This file contains the detailed results of the benchmark comparisons between the Apogee 2.3 and compilers from various competitors, including: SunPro 3.0 (C and FORTRAN) Fujitsu F90 1.0 (C and FORTRAN) GNU C 2.3.3 (C only) CenterLine 1.0.4 (C only) The benchmarks were run in May 1994 on a SPARCstation 10 Model 30, with the following characteristics: SPARCstation 10 Model 30 36 MHz TMS390Z50 CPU with integrated FPU (single CPU) Primary Cache: 20 KB Instruction, 16 KB Data (on chip) Secondary Cache: None Memory: 32 MB Operating System: SunOS 5.2 (Solaris 2.2) File System: UFS Disk Subsystem: Fujitsu M2654SA (2GB) SCSI System State: Multi-user, single user logon Machine and OS choice were based on the following considerations: The SunPro 3.0 and Fujitsu compiler suites are only available on Solaris 2, so we chose that OS. The most recent Solaris 2 release we have in house which is a customer-available, generally functional release is version 2.2. We have a SPARCstation 10/30 (which was used) and a SPARCstation 10/512 capable of running that release of the OS. We chose not to use the (multiprocessor) 512 for two reasons: the presence of an external cache (1MB per processor in this case) causes very unreproducable performance runs; and not all compilers are capable of making effective use of multi-processors (though we are), and we didn't want to stack the deck in our favor. Interested customers are encouraged to contact us for comparison results on multi-processor machines. We have made every effort to achieve the best performance with everyone's compiler, including our own. We requested optimal flags and official results for the SPEC benchmarks from SunPro and Fujitsu, but neither gave them to us. We found makefile wrappers for Metaware's compiler (CenterLine's compiler is based on Metaware release 2.5), but were able to get much better results than those wrappers gave. We are using the latest released compilers from each vendor available to us on the date of the tests (there probably is a newer version of GNU's compiler available, but we haven't set it up). We welcome any suggestions for improvements in flags for any of the compilers. We especially welcome suggestions from the respective suppliers! We started with previous SPEC published benchmark results. We tried adding various flags documented in the various manuals. We ran numerous experiments trying various combinations of flags. The best result has been used in each case. It is our belief that these represent the best case, "apples-to-apples" comparisons achievable with our resources. Once the best combination of flags was determined, each benchmark was run three times, with the best result used. For SPEC, "wall-clock time" is reported, as usual. For Lawrence Livermore Loops we have reported the Harmonic Mean, as usual. All vendors had C compilers, so we were able to obtain comparable CINT92 numbers for all vendors. Apogee, SunPro and Fujitsu also offer FORTRAN compilers, which we used (along with the C compilers) for the CFP92 and LLNL (Livermore) FORTRAN Kernels. To generate the graph, the SunPro compilers were used as a baseline. Everyone's performance was divided by the SunPro performance, and then converted to a percentage of the SunPro numbers. Hence, SunPro always got exactly 100% of their own performance (by definition). As it turned out, all other compilers got worse performance on the aggregate CINT92, CFP92 and Livermore numbers, and Apogee beat everyone on each of the aggregate numbers. The lists below contain performance data and flags used for each benchmark. ============================================================================= Performance Summary =================== For all the summary numbers here, higher numbers are better. The CINT92 benchmark is designed to measure CPU performance on various "system" and/or "integer" computation. There are 6 benchmarks, all written in the C language. The results can be combined into an aggregate number labeled "SPECint92": Apogee: 50.4 SunPro: 46.7 Advantage: Apogee 8% Fujitsu: 41.9 Advantage: Apogee 20% GNU: 41.4 Advantage: Apogee 22% clcc: 40.0 Advantage: Apogee 26% The CFP92 benchmark is designed to measure CPU performance on various floating-point intensive computation. There are 14 benchmarks, 12 of which are written in various dialects of FORTRAN-77 and 2 of which are written in the C language. The results can be combined into an aggregate number labeled "SPECfp92": Apogee: 55.3 SunPro: 49.3 Advantage: Apogee 12% Fujitsu: 42.3 Advantage: Apogee 31% The Lawrence Livermore National Laboratories FORTRAN Kernels is designed to measure CPU performance on 24 loops, each of which is designed to model the computationally intensive kernel in a different floating-point intensive application. The benchmark is contained in one program which is written in a dialect of FORTRAN-77. The results can be combined using a Harmonic Mean or a Geometric Mean. The Harmonic Mean is reported here (the Geometric mean is shown at the end of this file): Apogee: 9.34 SunPro: 7.35 Advantage: Apogee 27% Fujitsu: 6.93 Advantage: Apogee 35% The lists below contain performance data and flags used for each benchmark. ============================================================================= =============================================== | SPEC 1992 Integer CPU Benchmark (SPECint92) | =============================================== For all times shown, lower numbers are better. The time is the wall-clock runtime of the benchmark (in seconds). Short labels are used as follows: "Apogee" represents the Apogee AC2.3 compiler suite "SunPro" represents the SunPro SC3.0 compiler suite "Fujitsu" represents the Fujitsu F90 version 1.0 compiler suite "GNU" represents the GNU C version 2.3.3 compiler "clcc" represents the CenterLine C version 1.0.4 compiler, which appears to be based on the Metaware v2.5 compiler. 008.espresso: Times: Apogee: 52.6 SunPro: 53.2 Advantage: Apogee 1% Fujitsu: 57.6 Advantage: Apogee 9% GNU: 52.8 Advantage: Apogee 0% clcc: 54.5 Advantage: Apogee 4% Apogee Flags: OPT=-O4 -K -w -Xintedge,T=ss10/30 EXTRA_LDFLAGS=-dn EXTRA_LIBS=-L/usr/ucblib -lbsdmalloc -lucb used src.alt/espresso.h.apx SunPro Flags: OPT=-fast -O4 -cg92 -Qoption cg -QrmW -Qoption cg -QrmX -Qoption cg -QrmR EXTRA_LIBS=-L/usr/ucblib -lfast -lucb EXTRA_LDFLAGS=-dn used src.alt/espresso.h.sni Fujitsu Flags: OPT=-O3 -Ka4,dalign,lib,TMS EXTRA_LDFLAGS=-dn EXTRA_LIBS=-lbsdmalloc -L/usr/ucblib -lucb used src.alt/espresso.h.apx GNU Flags: OPT=-O2 -w EXTRA_LDFLAGS=-Wl,-dn EXTRA_LIBS=-lbsdmalloc -L/usr/ucblib -lucb -lelf used src.alt/espresso.h.apx CenterLine Flags: OPT=-O5 -Hit=100 -Hoff=Strength_reduce -Hpcc -w EXTRA_LDFLAGS=-dn EXTRA_LIBS=-lbsdmalloc -L/usr/ucblib -lucb 022.li: Times: Apogee: 100.0 SunPro: 132.1 Advantage: Apogee 32% Fujitsu: 137.1 Advantage: Apogee 37% GNU: 169.0 Advantage: Apogee 69% clcc: 139.1 Advantage: Apogee 39% Apogee Flags: Profile-directed dynamic feedback is used to build the timed version. First a profiled version of the benchmark is built with: OPT=-O -w -p -dn The benchmark is then run with input.short/li-input.lsp, to produce profiling data. This is processed with the prof(1) command with: prof -cg xlisp > PROF_OUTPUT 2> /dev/null All C files are then recompiled jointly (which permits inlining across source files) using the profiling data with: OPT=-O4 -w -XT=ss10/30 \ -Xinllev=3,inline=%profile50,pstat=PROF_OUTPUT,join EXTRA_LDFLAGS=-dn SunPro Flags: OPT=-fast -O4 -dn -cg92 -Qoption cg -QrmW -Qoption cg -QrmX \ -Qoption cg -T5 -Qoption cg -QrmYxleval -Qoption cg -QrmYxlevarg \ -Qoption cg -QrmYevform -Qoption iropt -Mt500 Fujitsu Flags: OPT=-O4 -Ka4,dalign,eopt,lib,TMS -x - EXTRA_LDFLAGS=-dn GNU Flags: OPT=-O2 -w -finline-functions EXTRA_LDFLAGS=-Wl,-dn CenterLine Flags: OPT=-O3 -Hit=200 -Hic=25 -Hpcc -w EXTRA_LDFLAGS=-dn 023.eqntott Times: Apogee: 13.8 SunPro: 15.5 Advantage: Apogee 12% Fujitsu: 20.3 Advantage: Apogee 47% GNU: 27.0 Advantage: Apogee 96% clcc: 29.8 Advantage: Apogee 116% Apogee Flags: putpla.c compiled with: OPT=-O -DSYSV -w -K -Xxopt5 -Xkap -XKstdio all other files compiled with OPT=-O -DSYSV -w -K -Xxopt5 Everything linked with: EXTRA_LDFLAGS=-dn -Xkap -XKstdio used src.alt/preprocess.c.sol2 SunPro Flags: OPT=-fast -O4 -dn -cg92 -Qoption cg -QrmW -Qoption cg -QrmX \ -Qoption cg -QrmR -DSYSV EXTRA_LIBS=-lfast used src.alt/preprocess.c.sol2 Fujitsu Flags: OPT=-O4 -Ka4,dalign,eopt,lib,TMS -x - -DSYSV EXTRA_LDFLAGS=-dn used src.alt/preprocess.c.sol2 GNU Flags: OPT=-O2 -w -DSYSV -funroll-all-loops EXTRA_LDFLAGS=-Wl,-dn EXTRA_LIBS=-lbsdmalloc -L/usr/ucblib -lucb used src.alt/preprocess.c.sol2 CenterLine Flags: OPT=-O7 -Hit=100 -Hoff=behaved -DSYSV -Hoff=Globals_volatile \ -Hoff=Strength_reduce EXTRA_LDFLAGS=-dn used src.alt/preprocess.c.sol2 026.compress Times: Apogee: 99.9 SunPro: 101.3 Advantage: Apogee 1% Fujitsu: 105.3 Advantage: Apogee 5% GNU: 101.3 Advantage: Apogee 1% clcc: 102.0 Advantage: Apogee 2% Apogee Flags: OPT=-O -K -w -Xkap -XKstdio -XT=ss10/30 EXTRA_LDFLAGS=-dn SunPro Flags: OPT=-fast -O4 -dn -cg92 -Qoption cg -QrmW -Qoption cg -QrmX \ -Qoption cg -QrmR EXTRA_LIBS=-lfast Fujitsu Flags: OPT=-O3 -w -Ka4,dalign,eopt,lib,TMS -x - EXTRA_LDFLAGS=-dn GNU Flags: OPT=-O2 -w EXTRA_LDFLAGS=-Wl,-dn CenterLine Flags: OPT=-O7 -Hit=200 -Hoff=Globals_volatile -Hoff=behaved EXTRA_LDFLAGS=-dn 072.sc Times: Apogee: 67.0 SunPro: 68.9 Advantage: Apogee 3% Fujitsu: 69.8 Advantage: Apogee 4% GNU: 67.1 Advantage: Apogee 0% clcc: 69.6 Advantage: Apogee 4% Apogee Flags: OPT=-O -K -w -DSIMPLE -DSYSV3 EXTRA_LDFLAGS=-dn EXTRA_LIBS=-lform -lcurses SunPro Flags: OPT=-fast -O4 -dn -cg92 -Qoption cg -QrmW -Qoption cg -QrmX \ -cg89 -Qoption cg -QrmR -DSIMPLE -DSYSV3 EXTRA_LIBS=-lform -lm -lcurses -lc Fujitsu Flags: OPT=-O4 -Ka4,dalign,eopt,lib,TMS -w -DSIMPLE -DSYSV3 EXTRA_LDFLAGS=-dn EXTRA_LIBS=-lform -lcurses -lm GNU Flags: OPT=-O2 -w -traditional -DSIMPLE -DSYSV3 EXTRA_LDFLAGS=-Wl,-dn EXTRA_LIBS=-lform -lcurses -lm CenterLine Flags: OPT=-O3 -w -Hit=100 -DSIMPLE -DSYSV3 EXTRA_LDFLAGS=-dn EXTRA_LIBS=-lform -lcurses -lm 085.gcc Times: Apogee: 133.9 SunPro: 135.4 Advantage: Apogee 1% Fujitsu: 166.9 Advantage: Apogee 24% GNU: 129.6 Advantage: GNU 3% clcc: 162.6 Advantage: Apogee 21% Apogee Flags: OPT=-O -Xc=tolerant -w EXTRA_LDFLAGS=-dn ALLOCA= used src.alt/stddef.h SunPro Flags: OPT=-fast -O4 -dn -cg92 -Qoption cg -QrmW -Qoption cg -QrmX \ -Xa -Dalloca=__builtin_alloca used src.alt/stddef.h Fujitsu Flags: OPT=-O4 -w -Ka4,dalign,eopt,lib,TMS EXTRA_LDFLAGS=-dn used src.alt/stddef.h GNU Flags: OPT=-O2 -w EXTRA_LDFLAGS=-Wl,-dn used src.alt/stddef.h CenterLine Flags: OPT=-O3 -w -Hit=100 EXTRA_LDFLAGS=-dn used src.alt/stddef.h ==================================== Geometric Mean for CINT92 (SPECint92) ==================================== SPECint92: Apogee: 50.4 SunPro: 46.7 Advantage: Apogee 8% Fujitsu: 41.9 Advantage: Apogee 20% GNU: 41.4 Advantage: GNU 22% clcc: 40.0 Advantage: Apogee 26% ============================================================================= ===================================================== | SPEC 1992 Floating Point CPU Benchmark (SPECfp92) | ===================================================== For all times shown, lower numbers are better. The time is the wall-clock runtime of the benchmark. 013.spice2g6 Times: Apogee: 685.0 SunPro: 771.7 Advantage: Apogee 13% Fujitsu: 831.9 Advantage: Apogee 21% Apogee Flags: OPT=-O4 -w -Xkap -XKinline=indxx -Xsave -XT=ss10/30 EXTRA_CFLAGS=-DUNDERBAR -DSYSV EXTRA_LDFLAGS=-dn SunPro Flags: OPT=-fast -O4 -cg89 -dn -Qoption cg -QrmR -Qoption cg -QrmW \ -Qoption cg -QrmX -Qoption iropt -algebra,reassoc EXTRA_CFLAGS=-DUNDERBAR=1 -DHAS_BCOPY=1 -dalign -fsingle EXTRA_LIBS=-L/usr/ucblib -lucb Fujitsu Flags: OPT=-X7 -fs -O3 -Kdalign,eval,nounroll,TMS EXTRA_CFLAGS=-DUNDERBAR -DSYSV 015.doduc Times: Apogee: 34.3 SunPro: 46.1 Advantage: Apogee 34% Fujitsu: 49.5 Advantage: Apogee 44% Apogee Flags: OPT=-O -w -Xkap -XKinline=si -XKinff=si.f EXTRA_LDFLAGS=-dn SunPro Flags: OPT=-fast -O4 -cg92 -dn -Qoption cg -T5 -Qoption cg -QrmR \ -Qoption cg -QrmW -Qoption cg -QrmX Fujitsu Flags: OPT=-X7 -fs -O2 -A A -Kdalign,eval,nounroll,TMS 034.mdljdp2 Times: Apogee: 110.3 SunPro: 126.3 Advantage: Apogee 14% Fujitsu: 137.0 Advantage: Apogee 24% Apogee Flags: OPT=-O -w -Xfcm2,mopt4 EXTRA_LDFLAGS=-dn SunPro Flags: OPT=-fast -O4 -cg92 -dn -Qoption cg -QrmR -Qoption cg -QrmW \ -Qoption cg -QrmX Fujitsu Flags: OPT=-X7 -fs -O3 -A A -Kdalign,TMS -x - 039.wave5 Times: Apogee: 101.5 SunPro: 105.5 Advantage: Apogee 4% Fujitsu: 116.0 Advantage: Apogee 14% Apogee Flags: OPT=-O -w -Xkap -XT=ss10/30 EXTRA_LDFLAGS=-dn SunPro Flags: OPT=-fast -O4 -cg92 -depend -dn -Qoption cg -QrmR -Qoption cg -QrmW \ -Qoption cg -QrmX -Qoption cg -Qpipe-unroll:6 Fujitsu Flags: OPT=-X7 -fs -O2 -A A -Kdalign,eval,TMS 047.tomcatv Times: Apogee: 40.0 SunPro: 47.2 Advantage: Apogee 18% Fujitsu: 66.9 Advantage: Apogee 67% Apogee Flags: OPT=-O -w -Xkap EXTRA_LDFLAGS=-dn SunPro Flags: OPT=-fast -O4 -cg92 -dn -Qoption cg -T5 -Qoption cg -QrmR \ -Qoption cg -QrmW -Qoption cg -QrmX -Qoption cg -Qpipe-group=1 Fujitsu Flags: OPT=-X7 -fs -O3 -A A -Kdalign,eval,preex,TMS 048.ora Times: Apogee: 64.3 SunPro: 68.0 Advantage: Apogee 6% Fujitsu: 66.8 Advantage: Apogee 4% Apogee Flags: OPT=-O -w -Xfcm2,kap -XKinlc EXTRA_LDFLAGS=-dn SunPro Flags: OPT=-fast -O4 -cg92 -dn -Qoption cg -QrmR -Qoption cg -QrmW \ -Qoption cg -QrmX Fujitsu Flags: OPT=-X7 -fs -O1 -A A -Kdalign,TMS 052.alvinn Times: Apogee: 64.9 SunPro: 69.2 Advantage: Apogee 7% Fujitsu: 87.5 Advantage: Apogee 35% Apogee Flags: OPT=-O -w -Xkap -XKarl=4 EXTRA_LDFLAGS=-dn SunPro Flags: OPT=-fast -O4 -cg92 -dn -Qoption cg -QrmR -Qoption cg -QrmW \ -Qoption cg -QrmX -Xa -I/l4/SC3/SUNWspro/SC3.0/include/cc Fujitsu Flags: OPT=-O4 -Ka4,dalign,lib,eopt,popt,GREG,TMS -x - 056.ear Times: Apogee: 349.9 SunPro: 349.0 Advantage: SunPro 0% Fujitsu: 386.9 Advantage: Apogee 11% Apogee Flags: OPT=-O -w -Xkap -XKinll=4 -XKarl=2 EXTRA_LDFLAGS=-dn SunPro Flags: OPT=-fast -O4 -cg92 -dn -Qoption cg -QrmR -Qoption cg -QrmW \ -Qoption cg -QrmX -Dfloat=double -Xa Fujitsu Flags: OPT=-O4 -Ka4,dalign,lib,eopt,popt,GREG,TMS -x - 077.mdljsp2 Times: Apogee: 105.2 SunPro: 116.9 Advantage: Apogee 11% Fujitsu: 118.6 Advantage: Apogee 13% Apogee Flags: OPT=-O -w -Xfcm2,mopt4 EXTRA_LDFLAGS=-dn SunPro Flags: OPT=-fast -O4 -cg92 -dn -Qoption cg -QrmR -Qoption cg -QrmW \ -Qoption cg -QrmX Fujitsu Flags: OPT=-X7 -fs -O2 -A A -Kdalign,eval,TMS 078.swm256 Times: Apogee: 295.4 SunPro: 286.5 Advantage: SunPro 3% Fujitsu: 336.1 Advantage: Apogee 14% Apogee Flags: OPT=-O -w -Xkap -XKag=a EXTRA_LDFLAGS=-dn SunPro Flags: OPT=-fast -O4 -dn -Qoption cg -QrmR -Qoption cg -QrmX -depend \ -Qoption iropt -O4,fusion -Qoption cg -T6 Fujitsu Flags: OPT=-X7 -fs -O2 -A A -Kdalign,eval,preex,TMS 089.su2cor Times: Apogee: 285.5 SunPro: 338.3 Advantage: Apogee 18% Fujitsu: 406.0 Advantage: Apogee 42% Apogee Flags: OPT=-O -XT=ss10/30 -Xalnref0,kap -w -XKinl -XKinll=1 -XKind=4 -XKag=a EXTRA_LDFLAGS=-dn SunPro Flags: OPT=-fast -O4 -cg92 -dn -depend -Qoption cg -QrmR -Qoption cg -QrmW \ -Qoption cg -QrmX -Nl200 Fujitsu Flags: OPT=-X7 -fs -O3 -A A -Kdalign,eval,preex,TMS 090.hydro2d Times: Apogee: 300.9 SunPro: 293.2 Advantage: SunPro 3% Fujitsu: 329.3 Advantage: Apogee 9% Apogee Flags: OPT=-O -w -Xkap -XKinlc -XKr=2 EXTRA_LDFLAGS=-dn SunPro Flags: OPT=-fast -O4 -cg92 -dn -Qoption cg -QrmR -Qoption cg -QrmW \ -Qoption cg -QrmX -Qoption iropt -par Fujitsu Flags: OPT=-X7 -fs -O2 -A A -Kdalign,eval,TMS 093.nasa7 Times: Apogee: 317.7 SunPro: 492.5 Advantage: Apogee 55% Fujitsu: 520.2 Advantage: Apogee 64% Apogee Flags: mxm.f compiled with: OPT=-O -w -dalign other files compiled with: OPT=-O -w -dalign EXTRA_CFLAGS=-DUNDERBAR -DTIMES EXTRA_FFLAGS=-Xkap -XKag=a Everything linked with: EXTRA_LDFLAGS=-dn SunPro Flags: OPT=-fast -O4 -cg92 -dn -Qoption cg -QrmR -Qoption cg -QrmW \ -Qoption cg -QrmX -Qoption iropt -par EXTRA_CFLAGS=-DUNDERBAR=1 -DTIMES Fujitsu Flags: OPT=-X7 -fs -O -A A -Kdalign,eval,preex,TMS -x - EXTRA_CFLAGS=-DUNDERBAR -DTIMES 094.fpppp Times: Apogee: 159.7 SunPro: 174.7 Advantage: Apogee 9% Fujitsu: 323.0 Advantage: Apogee 102% Apogee Flags: OPT=-O -w -Xmopt2 -XT=ss10/30 EXTRA_LDFLAGS=-dn SunPro Flags: OPT=-fast -O4 -cg92 -dn -Qoption cg -T5 -Qoption cg -QrmR \ -Qoption cg -QrmW -Qoption cg -QrmX -Qoption iropt -plus_iv Fujitsu Flags: OPT=-X7 -fs -O0 -A A -Kdalign,preex,TMS =================================== Geometric Mean for CFP92 (SPECfp92) =================================== SPECfp92: Apogee: 55.3 SunPro: 49.3 Advantage: Apogee 12% Fujitsu: 42.3 Advantage: Apogee 31% ========================================================================= ================================================== | Harmonic Mean and Options used for | | Lawrence Livermore National Laboratory Kernels | ================================================== Harmonic Mean and Geometric Mean are MFLOPS. Higher numbers are better. Harmonic Mean: Apogee: 9.34 SunPro: 7.35 Advantage: Apogee 27% Fujitsu: 6.93 Advantage: Apogee 35% Geometric Mean: Apogee: 14.1 SunPro: 8.5 Advantage: Apogee 66% Fujitsu: 7.9 Advantage: Apogee 78% Apogee Flags: -O -Xkap,T=ss10/30 SunPro Flags: -fast -O4 -cg92 -dalign -depend -Qoption cg -T6 \ -Qoption cg -QrmR -Qoption cg -QrmW -Qoption cg -QrmX \ -Qoption iropt Fujitsu Flags: -O2 -KTMS -X7 -fs =========================================================================